Lip Service or Actionable Insights? Linking Student Experiences to Assessment, Accountability, and Data-Driven Decision Making in Higher Education

Purpose Over the last three decades, the general public and policymakers have become increasingly focused upon the issues of accountability and data-driven decision-making within the postsecondary context (Altbach et al., 2005; Paulsen & Smart, 2001). More recently, subtle changes in the landscape of higher education have brought a new focus to accountability and with it “increasing pressure from various constituencies to demonstrate its effectiveness in measurable terms” (p. xi). Traditionally, the focus of assessment in higher education has been on student outcomes. More recently, however, assessment activities have broadened to include resource allocation, accreditation standards, comparisons with peer institutions, and satisfaction of students, faculty, and administrators (Ory, 1992; Shuh & Upcraft, 2001). The arguments supporting or challenging this movement toward accountability (and its cousins “assessment” and “data-driven decision-making”) often reduce to polarized and politicized ideological beliefs about the purpose of higher education and the mechanisms through which the academy can maintain (or develop, depending on your point of view) educational excellence and operational efficiency. Nonetheless, underlying the argument on all sides is the assumption that educational quality is likely to be improved when decision-makers (at all levels) develop policies and implement practices informed by relevant assessment data. Assessment is considered the vehicle to determine which of those policies and practices lead to improved student experiences, greater learning, and/or higher satisfaction. However, this assumption regarding the value of accountability, assessment, and data-driven decision-making remains largely untested. Therefore, we use empirical data from senior administrators at 57 diverse institutions across five states to examine the extent to which assessment and data driven decision making shape the experiences of nearly 9,000 first-year students. Specifically, this paper addresses two main questions: 1. Which institutions (and which divisions therein) are employing which types of assessment and data-driven decision-making regarding students’ first year of college? 2. To what extent does institutional adoption of assessment and data-driven decisionmaking correspond to levels of first-year student engagement and learning gains?